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ABSTRACT 

* The effects of individualized education progra^n (lEP) 

monitoring strategies on 20 special education teachers' long term. and 
short term estimates of student achievement, theix satisfaction with 
students' programs', and their modifications in students' programs was 
examined. Teachefs }9ere randomly assigned t*^,an lEP monitoring 
treatment: short term goal measurement (STGM) or long term go^l 
measurement (IiTGM). h^ly^s revealed that the teachers' long term 
estimates of student' affchievement were unrealistiially high and 
similar in *botb treatment groups. Their short term estimates wer^ ^ 
conservative, but more optimistic and realistic for the STGM group. 
Additionally; the STGM teachers were more satis.fifed with their 
students', programs a^d introduced instructional program changes less 
frequentry,. (Author) ' * ^ ' 
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Abstract - ^ ' . « 

• , ' . ^ » _ * 

* * * * # » ^ 

The study examined the effect of lEP monitoring strategies on 

teachers' long-term and short-term estimates of student afiftievement, ^ 

their satisfaction with s,tudents' programs , '^and their mtdf ficatiorts in 

students' ■ programs . Subjects were, 20 special education teachers, each 

of whom, sel^ct^ed three to four students for participation iri the study. 

Teachers were assigned randoirrV^ to an lEP moni toringi#treatment : Short- ^ 

Term^Goal Measurement (STGM) or l^ong-Term Goal Measurement (LTGM).^ ' 

Analyses revealed that the teachers' long-term estimates of -student 

achievement were unrealistically^high .and similar in both treatment groups. 

Yet, their short-term estimates were conserva^tv£ , but more optimistic 

^nd rec^stic for the STGM-group. Additionally , it|[ie STGM teachers were 

more satisfied with thei r jstudents ' programs and introduced-instructional . 

•program changes less frequen^tly. Possible explanations and implications 

for lEP monitoring strategies are discussed, ^ 



* ^ 



The Effect of lEP Monitoring Strategies on Teacher Behavior 



c 



Gathering and usir]|g information, to, s^ape and .reshape? instructional 

y 4' ' ' * • * 

programs that meet, indiy.idual student needs is an esse.ntial , if not 

the "most essential, task of teaching. What^k'inds of information do ^ 

teachers gather? Salmon-Cox (1981) reported, that teachers prefer in- 

formal techniques, including observa±ion\ teacher:-made tests, and 

"interaction" with 'Siudertts,' for gathering inforraatton on S'tudent ' 

achievement. Teachiers ^infrequently described formal, standardized'* 

test ijfiformatlon as useful in reshaping instructi,onal programs. ^ 

Salman-Cox noted that teachers seemed to prefer "continuous movies, 

in color with sound." - - 

# *• » 

^ An ap.proach'to data gathering that provides teachers with 'frequent, 

^Ibiet not continuous., pictures of student achievement is' direct and 
< *\ 

frequent measurement of student performance using systems variously 
describ.ed a»s ConJtiVuous Evaluation of IfP Goals (Mirkin, De'^no, Fuchs, 

Wesson, Tindal,' MarSton, &'Kuehnle, 198V), Data-Based Program Modifi- 

^' ' ^ ' ' — - ^ . 
ca-tion (Deno &'Mirktn, 197?), Exceptional Teaching (White & Haring, 

1980), 'and-^Precision Teaching (Lindsley, 1971). Models of direct and 

frequent measurement i nclxide procedures- for. goal selection, measurement 



of goal achl'evejnent, and -data analysi-s* to^heTp teachers .decide whether * 
'and how to 'restructure, curriculum to bfe consistent with the ,needs of 
individual stude'nts.. - • ^' \, ' " . ' 

ATthdUgf) the different system's. foe collecting direct and frequent 
measurements of student perfoijmance are 'siirrilar in. their basic approaches 
ta 'Obta'intng; pictures of student achi evement, they differ in their 
recommendations^ about the types* of 'data to collect, the format of graphs 



on which data are record^d< the^ways In which dala are to be utilized,. 

ajid the frequency with which dat^ are collected. The [Purpose of 

the pres*ent study was to determine the. effects of two variations of V 

dirtct and frequent measupfement on teacher decision making. Specifl- 

•cally, the study addressed the following questions: 

(a) Qo teachers' long-term and 'shortrterm estimates of ; 
student atchieveme/it dif/er with^l^he two variations- 
of. data collection an'd 'utiilization procedures? 

' (b) -Does teacher satisfaction cliffer with^the two procedures? 

(c) Do teacher modifications in programs differ with the two 
procedures? , n- . 

Method ' . - • . 



Subjects . . . » , ^ 

/Twenty special education resource teachers from a ijiidwestern 
metropolitan area voluftteerecf to serve as subjects in the study. 
These teachers (2 males and 18 fer/iales) had taught school for an 
average of 9.5 years (SD = 5.9). E^ich teacher selected three to 
four students from his/her caseload, resulting in a student sample 
of 73. boys and 15 girls. The >students | *medn age -was 10,3 years^ 
(SD =1.9); their mean grade level wa^ 3.9. • . 
Materials " ' ' • 

* Training manuals . Teacher training ma-nuals for two e:xperimental 
conditions werfe written; expressly for the study. .Both manual's consisted 
of six chapters on the assumptions of systematic vnstruc^^ion, and pro- 
c€dures for placing students, setting long-te>m goals , measuring 
.students, recdrding and graphing, data, and changing^sfudentipro^'rams . 
These materials •were self-instructional and each chapter concKided 
with mastery test. ' Two chapters, "Measuring Students.'^ and ' "Recording 



and Graphing Data," di ffered .for the two groups; these chapters reflected 

the experioiental 'treatirients described b§low. * . 

Word cards . Word cards organized in instructional units from 

each student's existing reading materials were -prepared for the teachers 

and were employed as fTashcards in the measurement procedure. 

Dependent Measures ' > \ ^ ' 

~ — , . - - 

Two\ types of dependent measures were employed. The first type 

v/as an interview checklist on which teacher decision-snaking information 

was recorded, including (a) how, why, and when program adjustments were* 

*made, (b') teacher re-estimates of long-tefm goals, and (c) teacher 

shcJrt-term estimates of student performance collected each time a * 

teacher introduced a program change (see Teacher Interview Questions 

in Appendix). Thq second type of dependent measure was a teacher ranking 

form on .which teachers selected and rank ordered, for each student, 
- • ^^^^^^ 

five effective program changes from among eight i nstructional , eight 

; f • ^ * 

md'ti vational, and eight administrative anS physical arrangement alter- 

natives (see Teacher Ranking Fornj in Appendix). 

Prod^dure 

Experimental conditions . Teachers were assigned randomly to one 
of two experimental treatment groups .for the purpose of measuring stu- 
dent progress: Long-Term Goal Measurement (LTGM) or Short-Term Goal 
Measurement (STGM'). In the LTGM condition^ teachers tested students*' • 
reading performance by administering a,30-second word recognition test 
•compri^^ed of 25 wprdS randomly selected from a large set of vocabulary 
words to be introduced within the 12-week period. -At each measurement 
session teachers .were required to graph the student^s performance; on ^ 
the sixth throTigh ninth .days, they were required to review the graphed 



data, 'if these. data indicated that* ppogr.ess was inadequate, then the 
teachers wer6 to introduce an adjustment in the prograp in an pittempt 
to improve the effectiveness of the instruction. After 10 days, 
teachers were required to make an adjustmeht if one had not been made 

3 ^ • — • ' , 

'prewiously. This routine 'was repeated several times in the 12-week 
-experimental period. ^(See Figure 1 for exaniple of a graph,) 



Inserf Figure 1 about here 



In the STGM group, teachers- tested students' readin^g performanJ:e 
by administering a 30-second wtfrd recognition test cojnpri-secl .of the 
hew vocaJDUlary words introduced in the current instructional period 
plus words sampled from precedi ng ''stories , for a total of 25 words. 
^Teachers graphed the-stu^nt^s performance and compared that perform- 
ance agains^a short-term aim line, whi-ch the teachers drew on a graph 
each time, a 'new short-term aoal was es|:ablished» Teachers were asked 
to review the graphs f re^fffrently to determine when to move to the, next 
story and/or when to make a program adjustment. This routine continued 
throughout the 12-week period. (See Figure 2 for example of a graph.) 



Insert Ftgure 2 about'here 

. . 

Training . At the first of two ^h hour^sessions, teachers'were. , 
trained in procedures for pTacing students in'their curricula. These 
procedures r'equired students to V^ad», in the-currently employed basal - 
reading series, tjiree one-minute. samples on each of the t^iree -highest 
levels'at which the^ student coul d read approximately 30-49 words per 
minute *(*wpm), with fewer than eight errors for poor readers, and 50-99 



wpm with fewer than eight errors for better readers. After three 
days* of data collection, teachers were to compute a median wpm and .^^ . 
median errorsLper minute for each level, and place the student at 
the highest level at which the above criteria were met. 

Having been trained in- this placement procedure, 'teachers im- 
pl^mehted it wh^n they returned to their schools. Although teachers 
had been advised to- place a stirdent in the highest level at which 
the'student met the performance standard, they often Qsed other pn- 
tei:,ia such as previous placement^, intuition, and logistical feas.ibiTi-ty 
of placements. Between tra idling session's, teachers also completed 
re&ding and answering questions \n the training manual s- 

.* At the second training session, training ijianual mastery tests 
were scored. . Additionally, teachers set lohg-'term (1 2-week) goals, 
as instructed in the manual, and submitted a .list of all vocabulary 
words included in ^those long-term goals. These words were made into 

• * - 

wordjCard packs.- . " ' " • , 

' " Teachgr visits . One week after 'the second trainin.g session, a 
graduate research assistant (RA) delivered the word packs to each 
teacher and helped the teachers set up student graphs. The teachers 
then began to implement -the measurement strategies.- 
/ An RA was assigned to- each' teaclier; over the 12-week treatment 
period, RAs made weekly 10- to 20-minute visits to their assigned ^ 
teachers. During each of those visits, the" RAs provided additional 
training'as^required, and recorded information on the interview check- 



list. When completing'the^checklist, ttie.-RA determined 'whether the , 
teacher wished to revise the long-term goal prediction and, if so, 
what the. new LTG was^ The RA also determined whether the teacher 
had'made-a change in 'th^ student 's_ program, and whether the^ change ^ * • 

V 

was an instructional, motivational, or physical arrangement change. 
Further if a change wa^ade, the RA recorded the teacher's new 
prediction of ttie student's short-term achievement (i.e., ttfe student's • 
midian level, of words corre'ct and errors over the new prograpi phase) . 
Finally,' the RA noted the teacher's judgment of the effectiveness of * 
tite previous week's program for each student. At Weeks 3, 6, 9, and 
•12, teachers completed the Teacher -Ranking Forms. 

V • 

Res ul ts 
* ^ — - 

Long-term' and Shdrt-te rm Estimates of Student Achievement 

— : ' ' ^ 

Long-term estimates . The mean number of revisions in long-term « 
goals' was comparjed 'for the STGM and the LTGM conditions; a t test 
revealed no significant .di fference'^between measurement groups. 
'Teachers typically made from one 'to three revisions in the students' 
LTGs . ' * • . . - 

> * 

When' goal estimates wer€ averaged across the 12 weeks, the number 
of \i/ords that the teachers fcia{i .predicted would be mastered was the 
same for'the STGM and'LTGM conditions. The initial and final predictions 
of the number of words that would be mastered also were compared for the 
two measurement conditions; a t test revealed a statistical-ly signifi- 
cant difference for the initial predictions (t = 2.08, £ = .04). The 



means in Tajjl^l indicate that the LTGM teachers initially predicted 

a greater number of words to be mastered than did the STGM teachers. . 

However, by the last week in the study, most tTGM teachers had revised 

their predictions downward, rendering a, fi ricil^ mean Estimate more 

similar to that of the STGM teachers, wliose fi*nal mean prediction was 

> * 

.very cloie t;o their- original mean prediction* 



Insert Table 1 about here 



At the end of tife IZ^week experimental period, teachers reported 
the nflmber of words each, student had actually mastered i'n his/her / 

^ ' * K * 

curriculum; a- 1. test applied to^ese flata revealed no statistic^ly 
significant difference between *the 5TGM aad LTGM^ groups. Stfl^ents- 
in the ITGM group jpastered^ an average of 2Q3.1 wgrds (SD = 153.3); * 
students in the STGM group ma^teredvMi average of 164.8 words (SD = 

T31.0). ^ • ' " ^ " ^ - y ' 

The aci:uracy of teachers''- LTG predictions was de-fined as. the 
difference between the number of words actual.ly mastered and the average 

N V * ' 

V.I, 

LTG pre;iiction for each subject. These differeTTces were aggregate's for 
the STGM and LTGM groups and were subjeicted tb a t test, which revealed 
no statistically signifjcant difference. Teachers irf both groups 
predicted that students would master more words than they actually . 
did ^master . The STGM teachers over-predicted by an a^e)>age af 20.82 
.words {SD = 78,89>j LTGM* teachers over-predicted by an average of 9.71 
words (SD = 100.35). " ' ' . • 

Short-term estimates . Short- terra estimates were the teac^rs' - 



predicted median levels of words cprrect and errors over upcoming 
program phases'. These predicted nredian level s. of performance were 
aggregated 'for the STGM and the LTGM groups and were subjected to 
t tests;, the results revealed statistically significant differences 
for words correct and incorrect per minute (t = 4.38 and 3.11, £ 
= ,000 and ^003, respecti'vely ) . The STGM teachers predicted more 
WQrds correct and fewer errors than th^e LTGM teachers (see'Table 2). 

* Insert Table 2 about here 

y ' . » 

^ • An analysis *f the accuracy of these predictions also was performed 
Accuracy was defined as the difference bet!?!^ a teacher's prediction 
and the student's ol?taine?i score. This difference was calculated for 
each intervention and then^t^e average difference' per intervention was 
computed, for each measurement group. The .STGM teachers underpredicted 
the students! correct performance by mean of .21 words. The LTGM 
teacher;s underpredicted the student's correct performance by a mean of 
1,61 words. The STGM 'teachers' predictions were more accurate than 
those of LTGM teachers, t(60) ='4.1, £ =..000. The STGM teachers also 
predicted students' errors more acc'urateJf than LTGM teachers, t(50) 
f 5.1, £ = .0000, -with the STGM teachers underpredicting ^errors by an 
average of .07 words and ^h^ LTGM teachers underpredicting errors by 
an -average of •49 words, ' / ^ - ' . ' * 

Satisfaction wi th Students' Programs 

— — » 

' Each week teachers judged for each of their' stu^dents whether 

the previous week's program had been effective. There v/as a 



statistically signi ftcant .re>ation between tHe uieasurement treaJtment 
and the number of weeks during which programs ^ere judged effective 
•(x 29.12, p = .002); - The number, of weeks in wMch higheV percentages 
pf/STGM teachers judged their students* prograpns as effective was 
greater than that, for the LTGM teachers. 
Modifications in Students' Programs 

During the weekly visits, RAs. determined whether teachers had . 
made an ins tructional "change, a motivational change, or a physical 
arrangement change in their students' 'programs'. Aggre,gated over VJeeks 
1-3, Weeks 4-5, Weeks 7-9, and^Weeks 10-12, teachers made a greater 
percentage of instructional chang^tban any other type of change 
(se^e^t^Table 3). ' - ' . , 

Insert Table 3 about here 

Teachers jnade no change in the program from 5 to 12 times during 
the study. The percentages of teachers at each frequency level are 
displayed in Table 4. A chi square analysis revealed that greater 
percentages of the STGM teachers made no change in their students* 
program's (x2 = 48.5, £ ='.000). Approximately 83% of STGM teachers 
made no chwge in their student's programs 9 to 1-2 times, while 11.3% • 
of LTGM'^eachers made no change 9 to 12 times (see Table 4)*^ Table 5 ^ 
presents the percentages of teachefs making instructional, motivational, 
and physical arrangement changes at different frequency level.s. This 

table shows that, for changes that wer.e made, greater percentages of 

'^^^ ' ' ' ^ 

LTGW^han SIGMJteachers made more motivational changes (see Table 5). 

- • n 
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Insert Tables^ 4 and 5 abbut here . ' 
V, ^ ^ 

Figure 3 displays the percentages of teachers^ averaged over Weeks > 

3, 6, 9, and 12, that ranked instructional, motivational, or physical . 

arrangement changes as first, second, 1?hird, fourth, or fifth most 

effective. Inspection of this figure reveals that motivational changes 

* were selected least often as most effective for students; physical 
changes were selected least oHen as second through fifth most effective 

* " The greatest percentage of teachers' top three' sel ections were instruc-. 

tional changes, while the percentages of their fourth and.^fifth selec- 
tions were distributed between instructional and motivational changes. . 



Insert Figure 3 about here 

jCw 
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^ * Discussion 
Initially, the LTGM teachers predicted .that theiv students would 
master a greater number of words in 12 weeks than did the STGM teachers; 
however, they revised their estimates to render them more similar to 
those of the STGM teachers. In both measurement conditions teachers 
made one to thr^e revisions in their LTGs-; the final accuracy of those 

* 

long-term estimates was. similar, with bpth sets of teachers overpre- 

dicting the number of words that would be mastered during the 12-week 

study. Therefore, teachers'* long-term esti-mates of student achievement^ 

did not differ as a function of the typ^^ cfata collected. Both groups 

#• 

overpredicted Tong-term goals. ' 

While teachers' long-term estimates of student growth" tended to be 

X . ■ . 

15 m 



unreal istica 117 high, their short-term estimates were' conservative . 

Both groups ^ teachers underpredicted the number of correct words 

students would read di/r>ng the upcoming program phas'es. However, the 

STGM tea'chers ptredicted that their students would read more y/ords 

correct and make j^wer errors than the^LTGM teachers predicted for 

their students. Further, ,STGM^ teachers ' predictions were more accurate. 

Consequently, it appears that teachers' short-term estimates differed 

as a function of the type of data^^hey collected. Teachers who monitored 

students' attainment of a series of short-term objectives, which changed 

approximately weekly, preda'cted performance more accurately -and less 

pessimistically than their counterparts who monitored students' progress. 

on a larger pool of material representing a 12-week goal. 

_ In addition to the STGM teachers predicting better and more realis- 

tic short-term performance than did -\the' LTGM teachers,^ the STGM teachers 

also judged more often that their students' programs were effective. 

^ < . 

The STGM teachers, then? felt more satisfied with their students' 

, programs as a function of the type of data they collected. This grea1 

satlsfactiorn may have been realistic given the fact that the STGM 

teachers .predicted short-tferm success more accurately. and higher *than 

did the LTGM teachers. ^ 

» s 

The STGM teachers' greater sati,sfaction appeiared to lead them to 
make fewer changes in, their s^tudents" programs. STGM teachers were 
free to modify programs as frequently as they deemed necessary to ensure 
*^^that^their students woul d breach goa>s . On the other hand, the LTGM 
teacTiers, who were required to. modify programs at least every lO^days, 
more, often reshei^ped their students ' programs . While teachers^ in both . 
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*i2 A. . • ■ ^ . : 

r * • 

groups most'frequently'made instructional changes in their students'^ 

' . " ' '4 

prograins,' the LK^M teachers made 3 greater percentage of motivational' - 
changes t^n did th'e^ STGM teachers. These motivational changes largely 
were directed at increasing student performance on the-nreasiwrement task 
rather than at sUt^stantially changing programs. For instance, LTGM 
teachers frequently changed their students' programs by introducing re- 

.inforce'mejit for increasing words correct on the 30-second word reading 
test. THerefore,' i t is not surprising that, despi te 'the* LTGM' s ^<;eater 
number of programmatic changes, students in both measurement ^condition5sr 
performed simi larly throughout the study and, by the end of the 12 
weeks, had mastered an equivalent number of words. ^ ^ 
While teachers' long-term estimates of student achievement did 
not differ as a function of their data collection and utilization 
procedures,' their short-term estimates of student achievement, their 
satisfaction with students' programs, and the number and type of modi- 
fications in their s^tudents' programs did differ. Teachers who monitored 
perfQORance on_ a series of short-term objectives and who were free, to , 
modyfy programs as frequently or infrequently as necessary: (a) were 

* more^realistic and less pessimistic about students* ^hort- term achieve- 
ment, (b)"were more satisfied with their students' programs, and (^c) 
made fewer modifications in their students^ programs with similar results 
in student progress as compared with teachers who monitored progress 
on long-term goals and who were required to modify student programs ^ 
at least every .10 days. On the basis of these findings, one might 
recoiranend to teachers that they monitor students' performance on. short- 
term, rather than long-term, objectives • 
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Table 1 

Means and, Standard Devieitions of Predictions and Actual 
Number of Words Mastered In ihe LTG 



Treatment Condition 



Pjredictipn 
X SD 



^Actual 
X . ^ SD 



LTGM 

Original 
Final 
STGM 

Original 
Final 



•292.5 295.7 

257.9 125.5 203.1 153.3 

187.4 128.0 •-- 

186.8 141.8 164.8 131.0 



Means and Standard Deviations of Teachers' Predictions of 
Rate Correct and Incorrect for Upcoming Interventions 





Rate 


Correct 

9- 


< 

Rate Incorrect 


Treatment Condition 


X 


SD 


X ' SD 


LTMG 


19.1 


5.2 


2.7 1.4. •• • 


STMG 


25.6 


6.6 


1.7 1.2 
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• Table 3 ^. ' , 1 


Percentages of Instructional, Motivational, and Physical 




Changes *in 3-Week Intervals 


, Weeks 


Percentages of Change?" ^ * " :* ^ 
Instructional Motivational Physicecl 


1-3 


CO 1/1 ' K 
Do 14 D 




"^^£0, 32 . '3 


• 7-9 * ' " 


60 32. ' 5 . \ 


10-12 * • 


49 33 . 13 - ■ ■ . 


t 4 




4 




« 

■ . . : \ 






21 



^ At-.. 
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Percentages of Teachers That Made No Changes in Their Students' 



Programs from 5 to .12 Times T?uri^^^t*he-^Studi( 



5 5^ — 

Trea tm"ent ' tonxii.ti on 



- ^ 



Number of Times 




LIMG 






5 




2.3 


5 
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^ Table 5 

Percentages of Teachers that Made Instructional^ Motivational, 
♦ and Physical Changes in Their Stadents' Programs from 
" iP'to 6 Times During the Study 







* 


Typff of Change 








Instructional^. 


Motivational ^ 


' Physical^ 


[dumber of Times 


LTGM 


STGM 


LTGM.- 


STGM 


LTGM 


STGM 


-T7<p> : 
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5.6 . 


27.3- 


'77.8 


\ 

70:5. ■ 


■ 88.9- 


1 


20.5 


47.2 


25.0 


13.9 


27.3 


8.3' 


" 2 


34.1 


27.8 


29.5 


8.3 


2.3 


2.8 

♦ 


• 3 


15.9 


. '11 .1 


11.4 


0 


0^ 


0, 


4 


' 9.1 
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' ■ 5'.8 
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/ 5 • ■ . 


J v. 4 
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» 0 
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0 


0 




2.3 




0 


« 

0 .. 


0 
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= 8.1 (£ = .2300) 
\^ = 22. Z (£ = .Q002). 
^■/■= 4.7'*-(£ = .0970)- 
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Figure 2. Example of^raph for Short-Term Goal Measurement condition. 
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Student's Nain6 



Teacher Interview Questions 



Date 





Has teacher 
made a 
change? 


Was the 
last 

change made 
effective? 


What level of t Errors 
performance \ 
do you expect t ; 
using tills 1 
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T)fi vmi 

JVJKX 

/think stu- 
dent will 
meet long 
term goal? 
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• • 1 


Week 1, . 






.0 1 
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Teacher Ranking Form 
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